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Introduction and Synopsis 


This paper describes an approach for solving the familiar dilemma of 
balancing audit needs with limited audit resources. It allows the auditor to look 
at the organization being audited in a way that quickly identifies the key systems 
and major control issues. The approach provides a method for a program-oriented 


audit of a department. 


The exploratory approach is a staged process in which data collection 
alternates with its analysis and evaluation. The initial data collection leads to the 
development of a first theoretical model of the organization. This model shows 
how the auditor first believes the program might logically be organized to do what 
it is supposed to do; it is called a system logic model. This first model is then 
analysed to identify what are thought to be key functions of the organization 
being audited. Further information is collected, with particular emphasis on these 
identified functions. Analysis and collection are alternated, and conclusions are 
displayed in more refined versions of the system logic model. This refined model 
can be used to explain to management of the organization being audited the 
auditor's perception of the organization. It helps to validate the auditor's under- 


standing of the organization. 
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Because comparable logic models are built up by the various 
specialists on the audit team, their models can be compared to give a balanced 
overview of the organization. Logic models can also be used in selecting the 
issues for detailed audit. Since the models visually display the relationships 
between various systems and processes, this facilitates the comparison of the 
need for detailed audit information being obtained on the various systems and sub- 


systems. 


Example 


The general purpose accommodation program of the Department of 
Public Works is used here as an example. It was selected for several reasons, 
including the convenient dimensions of this particular program-oriented audit and 
the fact that the audit team had to apply the full analytical process of the explo- 
ratory approach. Some background on the dimensions and characteristics of the 


Department and program are given. 


The Department of Public Works is the federal government's major 
realty organization. Its primary role is to provide government departments and 
agencies with accommodation economically and efficiently; it does this by 
acquiring, operating and maintaining buildings and by maintaining the govern- 
ment's central real property inventory. In 1977-78, the accommodation program 
expenditure was approximately 80 per cent of the Department's $830 million total 
program expenditure. Approximately 6,000 employees work in the accommodation 


program; most of the 1,336 employees in the professionai and technical services 
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program do work directly or closely related to the accommodation program. The 
general purpose accommodation activity accounts for about two-thirds ($442 


million) of the accommodation program's resources. 


The Theoretical Model of Accommodation Services 


As a first step in this approach, the auditor develops a system logic 
model of the general purpose accommodation program (Figure 1) based on reading 
and analysing program forecasts, the Estimates, annual reports and publications 
on private sector or real estate management. This model then shows how the 
auditor at first believes the accommodation program might logically be organized 


to do what it is supposed to do. 


An accommodation program would begin with a demand forecast for 
its services and a comparison of this demand forecast with a list of available 
inventory, thus establishing a net shortfall or surplus. From this comparison, the 
program can define its options, analyse them and make recommendations to meet 
the demand. The decisions would be made and the accommodation acquired, 
operated and maintained for the duration of the demand. Finally, the 


accommodation would be disposed of. This logic is displayed in Figure 1. 


Developing a Balanced Perspective 


A theoretical model such as Figure | is the basis for parallel audit 


activities by different specialists. One problem associated with conducting 
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Figure |: The first theoretical model of the way an accommodation provision 
business might logically be expected to operate. 


comprehensive audits of complex systems is that of developing a balanced pers- 
pective of the program being audited while using the required specialists as 
auditors. Specialists in different areas will usually perceive an organization in 
different ways. To overcome this difficulty the method described provides a basis 
for developing the needed common perspective, by displaying clearly where spe- 
Cialist perspectives differ. In the example, we describe first the development of a 
logic model for the program-oriented management perspective of the accommoda- 
tion program. This is followed by a description of how models would be made for 
the financial management perspective, the EDP perspective and, by implication, 
any other perspective. We then discuss how the common perspective will be 


developed. 


The Program-oriented Management Perspective Logic Model 


The auditor transforms the first theoretical model through a series of 
interim system logic models into a final logic model. Interviews will have validated 
the auditor's understanding of the accommodation program; advice will have cla- 


rified that understanding within the audit context. 


Figure 1 acts as the base on which successive "layers" of information 
can be added, piece by piece. The logic model will be subject to modification and 


addition throughout the whole of the audit, not just during the preliminary survey. 


A simplification of a typical final logic model is shown in Figure 2. 


The final logic model is to reflect the auditor's developed state of knowledge; help 
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identify the key activities, systems and controls; highlight the needs for specifi- 
cation of criteria; and provide a basis of comparison, allowing proper selection of 


issues for detailed audit. 


Exploring the Program Being Audited 


The auditor uses the first theoretical logic model of Figure 1 to make 
a tentative identification of the key systems. This identification directs attention 
toward what are initially perceived as probably important matters. At this early 


point, the key systems are identified, using Figure 1: 


- demand forecast}; 


- option analysis; and 


- acquisition. 


It may be found later in the survey that this first identification is faulty, but, 
even so, the model guides the auditor's exploration of the program being audited 


and increases understanding. 


Using the general approach, more data is collected with emphasis on 
the identified systems. Subsequently, there is a review of all the information and 
a reassessment of what appears to be important. Collection and assessment alter- 
nate as the logic model is elaborated until the audit team is confident that it 


properly understands the program being audited. 
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As an example of the build-up of understanding, the auditor learns that 
demand forecasting, although key to the general purpose accommodation function, 
is not a feasible expectation in the Department of Public Works. Demand for 
accommodation is determined by government policies and by creation of depart- 
ments, their relocation and degree of decentralization. The Department of Public 
Works is often the last to know. Hence, with demand in the hands of Government, 
good forecasting is not within the control of the Department. Therefore, the 
resolution of this issue is outside the scope of our audit. The important issue of 
economy in government remains, however, and this could be the subject of a govern- 


ment-wide study. 


The auditor learns that options analysis is, in fact, a critical function. 
Here, the first analysis has proved correct. As for what the options analysis pro- 
cess should do, criteria for this exist and were reported in our 1978 annual Report. 
Further, the auditor learns that the acquisition process, which is project man- 
agement, is not costly even though it is important. Capital costs represent about 
one-quarter of the total accommodation costs, and the actual acquisition process 
represents about one-tenth of that quarter. The auditor discovers that reliable 
feedback loops for operations/maintenance costs to building design and options 
analysis are essential to cost control. Such elements of cost are, in turn, essential 
in estimating life cycle costs -- one of the criteria in options analysis. One learns 
that control of operating costs is feasible largely only at the building design stage 
and that maintenance costs are controllable at the building design stage and 


throughout the occupation of the building. 
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This example demonstrates that increasing knowledge allows continuous 
modification of the theoretical model eventually to provide the final logic model 


with its functions and feedback control loops. 


The actual final logic model for the accommodation program was 
immensely complicated. It displayed dozens of interconnections among many 
management systems and sub-systems. For simplicity in this example, Figure 2 is 
represented as being the final logic model of the general management function of 
the accommodation program. The "completed" Figure 2 shows the auditor's under- 
standing of how an accommodation program would logically be expected to operate. 
Each box represents an identified key system that is a candidate for further audit 
examination. The visual display of the relationships between and among the key 
systems makes it possible for the audit team to make an informed, although qua- 
litative, assessment of the relative importance of each key system. The most 
important are agreed upon, and these are selected for further more detailed 


examination. 


In continuing our example, we examine only three systems: 


- options definition and analysis; 


- feedback of life-cycle operations/maintenance to options 


analysis; and 


- operations/maintenance. 
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Figure 2: A representation of the final logic model for the management perspective of the General Purpose Accommodation 


function. 


Two further uses of the logic model should be noted. The first is that 
the boxes on the logic model represent logical ways to group the audit criteria. 
This is particularly important where a number of auditors are working simulta- 
neously on similar entities. By having a basic logic model with similar criteria, it 
is possible to get inter-team uniformity. A second use of the logic model is as a 
basis for the working paper file system. In one example, three new and informative 
files could be created with the titles: Options Definition, Feedback of Costs, and 


Operations/Maintenance. 


Option Definition and Analysis 


The auditor specified and validated criteria for options analysis with 
management of the program being audited. Of these criteria, the most important 
was that requests for funds should be supported by accurate and complete docu- 
mentation. This criterion implied a need for life-cycle costing and accurate 


estimates of comparable market rents and rental potential. 


The options analysis function was reviewed by examining some recent 
cases. A cursory review at this stage of the audit showed three potentially 


damaging weaknesses: 


- life-cycle costing was omitted; 


- calculation of financing costs was inaccurate in one case; and 
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- revenue estimates were imprecise. 


At this point, the auditor can foresee that such weaknesses would have 
significant adverse effects on any building program and plans a detailed examina- 
tion of the options analysis function to ascertain whether these suspicions of weak- 
ness can be supported. At the same time, the auditor considers planning an audit 
of a recent acquisition to see if these suspected weaknesses had adverse effects in 
that case. This has to be done to show that weakness suspected in the general 
management process actually would lead to adverse effects. It is equally 
necessary to demonstrate the absence of compensating controls. In this latter 
case, the auditor must be satisfied that remedying the weakness in the general 


management is necessary to correct the problem. 


It was decided to audit a recent acquisition project. The primary 

objective of this examination was to identify and report on the consequences of 

all suspected weaknesses in the general management process of the accommodation 
program. The suspected weaknesses in the options definition and analysis function 
would probably have effects such as poorly estimated costs, inaccurate revenue 
forecasts, and user needs not met. If such deficiencies were to be found in the 
audit of the recently completed project, a second level of detailed examination 
would quantify the impact on cost of such deficiencies.* Further detailed examina- 


tion would be necessary to determine whether such deficiencies in the project 


* A secondary objective of such an audit would be to evaluate the manage- 
ment controls particular to the project. 
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were, in fact, caused by the weaknesses in the options definition function of the 
general management process. If it were to be subsequently proved that the weak- 
nesses in the options definition and analysis function caused or permitted such 
deficiencies, then a recommendation to strengthen and improve the analysis function 
would be made. This rigorous proving must be done to demonstrate that the 
correction of the weaknesses in the general management system is likely to prevent 
recurrence of similar deficiencies in other building projects. This level of intense 
analytical effort is required if eventual recommendations are to be effective 


when accepted and implemented. 


Conversely, suppose that the preliminary evaluation of the options 
analysis function revealed no potentially damaging weaknesses. The importance 
of the options analysis function would, however, require that substantive proof of 
its effectiveness be obtained for inclusion in the overall evaluation of the manage- 
ment processes. Consequently, an audit plan to verify the suspected strength of 
this important control would be developed. Such a plan might require that the 
planning stages of a number of major buildings projects be reviewed in detail to 


verify that the options analysis function had produced the desired results. 


Feedback of Operations/Maintenance Costs 


Assume that preliminary evaluation indicates to the auditor that 
operations/maintenance costs are not fed back to either design or options analysis. 
The auditor would then develop a plan to verify this observation during the execu- 


tion phase of the audit. In the detailed audit, documentation would be reviewed, 
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operating personnel would be interviewed, and procedures would be analysed for 
completeness and compliance. Finally, structured interviews with key managers 


in the organization being audited would include questions such as: 
- Is this control link a logical part of the process? 
- Is such a control link present? 
- Is such a control link practically and economically feasible? 


This approach ensures that management will not be surprised by the 
audit observations, an important step toward eventual action on the recommenda- 


tions. 


Were weakness of general management function to be proved, predictable 
adverse effects would become evident during the audit of the recent acquisition. 
That is, the auditor could expect to find higher operations/maintenance costs 
designed into the building. To examine this issue, the auditor would determine 
acceptable standard cost levels from private industry experience. If higher costs 
were found at the new building, the auditor would have to verify that these could 
be reasonably attributed to design and feedback issues. The auditor confirms the 
absence of compensating controls and again must be satisfied that a recommenda- 


tion to provide for feedback of these costs to design would prevent a recurrence. 
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Operations/Maintenance Costs 


While developing Figures | and 2, the auditor learns that operations/ 
maintenance costs are controllable to some extent (but not to an equal extent) in 


the building design stage. 


We leave to the reader the problem of drawing a theoretical model for 
co-ordinating building design and options definition and analysis. Such a model, an 
analogue of Figure 1, might represent how this co-ordination would logically be 
effected. This model would guide the exploratory interviews so that a final logic 


model for the design-options co-ordination function could be developed. 


Alternatively, building maintenance costs are more determinable and 
more economically controllable during building occupation than at the design 


stage, although design intended to minimize cost is important. 


Figure 3 is the theoretical model developed by the auditor to represent 
how the maintenance function would logically be organized. From users come 
demands for maintenance service that require evaluation and decisions about the 
acceptance or rejection of them. If the demand is accepted, choice of service 
mode becomes a further necessary decision. There are three possible service 
modes: Public Works staff, contract staff, and joint Public Works and contract 
staff. The chosen mode provides the required maintenance service. After pro- 
vision of such service, evaluation of service is fed back into the evaluation of 


demands and into future decisions on the selection of service mode. 
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Maintenance Service Evaluation 


Theoretical model for provision of maintenance services. Note the link with 
Figure 2 where the demands for service are recorded for use in the box "Service 
Demand Analysis". 


2 {ae 


From this theoretical model, four key control points and relevant cri- 


teria are immediately obvious to the auditor. 


Evaluation of demand for service. Maintenance managers should have 
standards for facilities maintenance so that they can evaluate demands for service. 
For the detailed audit, the auditor would plan to establish the existence and rea- 
sonableness of these standards. In default of either or both desired states, what is 


the result? 


Selection of service mode. Maintenance managers should have a 
decision procedure that selects the most economical mode. In the detailed audit, 
the auditor would plan to examine this decision procedure. If inadequate, do 


identifiable deficiencies result? 


Evaluation. This should be a routine function used by all levels of 


management for various purposes. 


Provision of maintenance services. Of the four key control points, this 
is the most probable location of an efficiency issue. If, for example, contract is 
the selected service mode, the auditor will plan to examine adequacy of pro- 


cedures used to ensure cost effective use of contract personnel. 


To continue this example, let us assume that the DPW staff service 
mode is the dominant mode of the Department. The briefest consideration of this 
service mode shows the provision of service as a highly probable candidate for 


eventual efficiency issue analysis. Again, an audit management decision must be 
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made. Should these lower level systems and their outputs be examined? The 
answer is yes only if the information is needed to support the overall opinion 


which is being developed. 


Suppose the decision was made to audit the Department of Public 
Works service mode. The auditor will develop a theoretical system logic model of 
the management function during the preliminary survey. This model is used to 
guide interviews, identify key sub-systems controls, and, when the model is 
complete, plan the detailed audit. Figure 4 is such a theoretical model of the 
management of Public Works staff providing maintenance services. Demands 
enter the system and must be balanced against the inventory of maintenance 
resources. Demand and inventory are presented to the work control system which 
provides input to first level supervision that will be responsible for carrying out 


maintenance tasks. Later, there will be a feedback of performance evaluation. 


There are evident links between this latest system logic model and 
other models. The demands for service should be analysed to determine the best 
level of inventory for maintenance resources. The service demands analysis is a 
function in Figure 2. There must also be a link with the cost accounting system 
displayed in Figure 5, the logic model for the financial perspective of the 


accommodation program. 


From Figure 4 the auditor sees that four key controls merit attention. 
All relate direct to an efficiency issue involving transforming of maintenance 


resources into maintained facilities. They are: 
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Figure 4: Theoretical logic model for management of Public Works staff providing 


maintenance services. The service demands link with their analysis, 

a function indicated in Figure 2 where the size of the maintenance resource 
inventory would be determined in the service demand analysis function. 
The work control system should be linked with the cost accounting system 
of Figure 5. 


spac, tiate GINS 2% 
were 
oft 


a 


i gicyiam © 
ee ee 

is )  & 

Pr) ee ae) 


Spikes 


- work control system for estimating, scheduling and costing; 


- audit of work control system; 


- first-line supervision that is active, trained and well supported; 


and 


- evaluation. 


With these controls identified at this level of generality, another audit 
management decision must be made: Should a detailed logic model for mainte- 
nance service be developed during the preliminary survey? Or should this be 
deferred until the detailed audit work? The answer will be based on consideration 
of factors including audit resource availability, scheduling, and estimates of the 
probable audit effort needed. Considering only the last factor, if the audit effort 
is small, the detailed logic model will probably be postponed. If the efficiency 
issue is vital to the comprehensive opinion and the audit effort will be large, the 
logic model extension to Figure 4 is necessary to plan the efficiency audit. This 
being so, the final logic model for maintenance services will be developed at this 


point. 


This sort of audit decision has to be made continuously during the 
survey stage of the program-oriented audit. Unless controlled closely, this 
approach to the audit of complex systems will be very costly in terms of audit 


resources. 
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Another and equally important audit management decision must be 
made -- how to approach this audit. Should audit of the recent acquisition project 
be done as a project of the accommodation audit? Should it be done separately as 
an audit of capital asset acquisition? In fact, the decision could well have been to 
audit more than one such major acquisition. Again, time, audit resource availability 
and scheduling must be considered. In the actual situation, separate audit was the 
final decision. This being so, the modelling cycle re-starts from a theoretical 
model of a capital asset acquisition and disposal activity. Each major auditing 
management decision is made from a point closer to the operating realties of the 
program under audit, and each decision moves the auditor still closer to a thorough 


understanding of the program. 


The Financial Management Perspective Logic Model 


In parallel with the development of the program-oriented management 
logic model, a financial management logic model for the general purpose accommoda- 
tion program is developed. This model highlights for possible audit the financial 
issues in the audited organization. We present now the financial auditor's approach, 
seen from the exploratory point of view. We do this to clarify the relationship 
between the established systems approach of the financial auditor and the program- 
oriented approach to the comprehensive audit of complex systems. Ideally, the 


financial and the management aspects of the survey should be completely inte- 


grated; however, this is not always practical. 
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In an actual survey, the audit team would have management specialists 
developing Figure | into Figure 2. At the same time, financial management spe- 
Cialists develop Figure 1 into a logic model for financial management of the 
general purpose accommodation program. Under the survey team leader, specialists 
or specialist groups would co-ordinate their efforts so that a balanced perspective 
is developed. This modified figure is shown as Figure 5 -- a theoretical model of 
the financial management functions which might be identified during the preli- 
minary survey. Many feedback loops are now shown, and some management 
functions have been split. The design and construction box of Figure 2 is split, as 
is revenue collection, an important component of building administration. All 
such modifications to Figure 2 are put in place to aid and establish the location 
and relevance of financial management functions in the accommodation program. 
The critical financial functions are heavily outlined in Figure 5 and will include: 
option definition and analysis; revenue collection; revenue forecast; project cost 
control; routine financial management; cost accounting; performance measure- 


ment; and management of the financial function. 


Any financial management function considered important to the eco- 
nomic, efficient and effective management of the accommodation program will 
be considered for audit. A function will be audited if information on the quality 
of systems supporting that function will be needed to support the overall compre- 
hensive audit opinion. Although it is unlikely that many of these functions would 
be audited as part of the attest of the Public Accounts of Canada, many of these 
would be material to the comprehensive audit opinion on the accomodation pro- 


gram. 
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Figure 5: Theoretical logic model of the financial management functions in the General Purpose 
Accommodation Program. The financial functions are in the boxes with the heavy outlines. 
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For example, performance measurement, revenue forecasting and cost 
accounting each provide important inputs to the options definition and analysis 
function. In turn, this analysis function must perform well if value for money is 
to be achieved. Consequently, these three financial functions would probably be 
audited in detail because their individual assessment would be needed to form the 
overall evaluation of the quality of financial and resource management of the 


accommodation program. 


EDP Perspective 


In an audit of the EDP function associated with the general purpose 
accommodation program, the generic similarity of approach is maintained. To 
guide the audit, the location of major EDP systems will be indicated on Figure 5; 
this is done to produce Figure 6. This will help the auditor identify the compo- 
nents of the EDP system whose proper operation is necessary for effective and 
economic management of the accommodation program. As with Figure 5, this 


will be done during the survey. 


A parallel audit management decision will determine if a specific EDP 
program should be audited. This decision, like similar decisions needed in non- 
EDP audit, will be affected by considerations of the expected use of the informa- 


tion in developing the overall comprehensive audit opinion. 


Consider, for example, the computer usage at three locations in the 


program as shown in Figure 6: 
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‘gure 6: Theoretical logic model for the major EDP systems of the Accommodation Program. 


The important EDP systems are outlined heavily and connected to their respective 
support functions. 
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- at option definition and analysis function - computer analysis; 


- at the building design state - computer analysis; and 


- at the cost accounting function - computer records. 


The first and the third are important for the overall economic acqui- 
sition and operation of major buildings. The second might be presumed to be 
adequate because of the wide availability of packaged computer software and 
hardware; it would not be audited further. In the EDP detailed audit, the first and 
third would be reviewed only to ensure that they did effectively what they were 
supposed to do. They would probably not be reviewed to determine whether they 
were themselves economic and efficient. This is because their costs are relatively 


insignificant in comparison with those of the accommodation program itself. 


Developing a Balanced Perspective 


The balanced perspective is easily obtained when the logic models 
developed by the various specialists use the same basic structure. The manage- 
ment perspective logic model of Figure 2 could be modified to resemble more 


Closely the logic models shown in Figures 5 and 6, the financial and EDP perspec- 


tives. 


In any case, an informed judgement can be made about the relative 
significance of various auditable issues identified in the different logic models. 


Such a judgement derives from a balanced perspective. 
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Actual Accommodation Logic Model 


Figure 2 is represented as being the final system logic model. It is 
not. The final logic model was immensely more complicated. The actual final 
logic model was used in the way we suggest that Figure 2 be used in this example; 
that is, from the final model, key controls and functions are identified for further 


audit examination. There were many more than the three described. 


During the actual survey, a large number of functions and information 
linkages were subjected to preliminary evaluation. From these many preliminary 
evaluations, the detailed audit program was determined. In fact, Figure 2 was 


developed during the audit and not during the survey. 


Conclusion 


Throughout this text, we refer to links between the various systems. 
These links make the overall system complex. This complexity, in turn, makes the 
comprehensive audit of such systems difficult to manage. The audit of complex 
systems is done by breaking down the work into fixed projects and assignments, 
each with specific goals and fixed resources and schedules. A satisfactory compre- 
hensive audit requires thorough coverage of significant issues. An economic audit 
achieves this with a minimum of overlap and redundancy in audit effort. The 
early identification of important links between systems provided by this approach 
ensures economy in audit resources. It is the argument of this paper that the 


exploratory approach is the best way to conduct an audit of complex systems. 
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